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, EDGE CASE 
Overview C RESEARCH 





cs Where i is the industry in general as a sh Pistes ? 


=m Beyond the SAE Levels 
e Role of human vs. technology 


= Industry trends for 2021 
e Role of standards 
e Technical challenges 
e Organizational challenges 
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EDGE CASE 

RESEARCH 
NHTSA lifts suspension of 
EasyMile vehicles 


Low Speed Shuttles 


= Low speed shuttles 

e Up to 15 passengers 

e Fixed route at perhaps 5-10 mph 

e Demonstrations in cities worldwide 
= Safety approach 

e Slow speed limits kinetic energy 

e Often a non-driver safety conductor 
= Example Mishaps 

e Shuttle hit by backing truck (Las Vegas, 2017) 

e False alarm emergency stop with passenger injury (Ohio 2020) 


Smart Columbus < . 
https://bit.ly/39ki41t 
By Cailin Crowe 

Updated May 19 2020, 10:30 a.m. EDT e-Published Feb. 27, 2020 
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EDGE CASE 
RESEARCH 


Nuro Gets First Commercial 





Parcel Delivery 


ia S 8 (2) 
Parcels to SEIS, houses Autonomous Vehicle Permit in 
e Short range delivery California 
- Roads, bike lanes, sidewalks Prepare yourself mentally to see a Prius driving itself if you live in the Bay Area. 
e Demonstrations in several cities 2 


= Safety approach 
e Early: trailing vehicle 
e Later: remote human 
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= Example Incidents — 
e Sidewalk bot blocks wheelchair ramp (Pittsburgh, 201 9) 





e Tension over use of sidewalk space 
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0 ° . EDGE CASE 
Driver-Monitored Automation CB escance 


NTSB: Tesla Autopilot, distracted driver caused fatal 
crash 
By TOM KRISHER | February 25, 2020 





= Automated driving of car or truck 
e Continuous driver supervision 
e OEMs in production already 
= Safety approach 
e Human driver monitors automation 
e Human driver responsible for safety 
= Example Mishaps 
e Multiple fatal Tesla crashes 
— Issue: driver complacency 
— Issue: under 10 seconds from OK to fatal crash 
e Tempe Arizona fatality in testing (Tempe, 2018) 


https://bit.ly/3bnk3EZ 
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. EDGE CASE 
Fully Autonomous Operation © RESEARCH 
Waymo’s robo-taxi service 
opens to the public in Phoenix 


f ¥ in 





= Fleet vehicles 
e Waymo robotaxis deployed a limited scale 
e Middle-mile trucks gained interest in 2020 
e Many players pushing hard in this area 

= Safety approach 
e Early: Human safety driver 
e Later: Human on-call if car asks for help 

= Example incidents 
e California reports indicate minor incidents in testing 


https://bit.ly/39j4yeC 
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EDGE CASE 
RESEARCH 





Industry Trends 





= Consolidation in the “race” to autonomy 
e It takes huge resources to succeed 
e Trend to OEM + ADS supplier teaming , 
e Smaller players fail, team, or acquired over time “= 

= Fully autonomous pivot toward freight 
e Low kinetic energy for last mile service 
e Middle mile highways less chaotic than urban 

m Shift of “SAE Level 3” vehicles to L3+ = 
e Strict L3 means human driver supervision ert. eee 
e OEMs shifting to L3+ with car safe stopping on its own 
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EDGE CASE 


Standards-Based Engineering Approach © RESEARCH 





SYSTEM 
Sct, 1 UL 4600 
DYNAMIC iso/PAS 
DRIVING 
FUNCTION : 

FUNCTIONAL ISO 
SAFETY 26262 
CYBER- SAE 
SECURITY 3061 
VEHICLE 
Saray La MUSs 





SaFAD/ISO 
TR 4804 


SAE 
21434 





NCAP 








Safety Beyond 
Dynamic 
Driving 
Environment & 
HIGHLY 

sage (Ses AUTOMATED 

VEHICLE 
Equipment SAFETY 
Faults CASE 
Computer UL 4600 
Security 
Basic 
Vehicle 
Functions 
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EDGE CASE 
RESEARCH 


https://bit.ly/3q7VCz 


2021 Technical Safety ae 


a Perception & prediction 
e Safety of machine learning-based functions 
e Need more than object motion tracking 
= Safety of Intended Function (SOTIF) 
e Drive/Fix/Drive iteration with lots of testing 
— Waymo: 6M test miles; 65K deployed miles 
e How will safety be argued for larger fleets? 
— Likely will involve UL 4600 concepts and safety cases 
= Getting from “works OK’ to “safe” 
e You can brute force the first few “nines” ... but not all of them. 
e Field feedback into safety cases 
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Developing Trust for Full Automation C RESEARCH 
= Still an open world with unknowns & changes 
e Want “Positive Risk Balance” (safer than human driver) 
e But ... 20 human driver responsible 
ws Use Positive Trust Balance TRUSTWORTHY POSITIVE RISK BALANCE 
e Engineering rigor ent en en a 
e Practicable validation 


e Strong safety culture 
.. and ... 


e Field feedback 


to handle SUFPFISes Engineering Validation Safety 
Rigor Culture 


= UL 4600 ties feedback to Safety Case © 2021 Philip Koopman 11 


BUILD IT RIGHT 
TEST IT RIGHT 
IMPROVE IT RIGHT 
LIVE IT RIGHT 

















Safety Arguments (Safety Case) 


EDGE CASE 
C RESEARCH 
= Claim — a property of the system 


e “System avoids pedestrians” CLAIM 


= Argument —- why this is true 


e “Detect & maneuver to avoid” 
oS fa) ee @ 


m Evidence — supports argument 
e Tests, analysis, simulations, ... 


= Sub-claims/arguments address 
complexity 


e “Detects pedestrians’ // evidence 
e “Maneuvers around detected pedestrians’ // evidence 
e “Stops if can't maneuver’ // evidence 
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Safety Performance Indicators (SPIs) Cg Reseaxcu 


= SPls monitor the validity of safety case claims (UL 4600) 


LAGGING Vehicle is Safe ~ 
METRICS 
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Examples of SPlIs C RESEARCH 
= “Acts dangerously’ is only one dimension of SPls 

e Violation rate of pedestrian buffer zones 

e Time spent too close per following distance math 
= Components meet safety related requirements 

e False negative/positive detection rates 

e Correlated multi-sensor failure rates 
= Design & Lifecycle considerations 

e Design process quality defect rates 

e Maintenance & inspection defect rates 


= Is it relevant to safety? =} Safety Case =} SPlis 





© 2021 Philip Koopman 14 


EDGE CASE 
RESEARCH 





2021 Safety Themes 


= Positive Trust Balance: : 
e Engineering Rigor, Validation, Feedback, Safety Culture 
e Standards-driven safety 
e Transparency 
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= Safety Performance Indicators (SPIs) 
e Continual improvement & updates 
e Field feedback: development; deployed 





= Scalability past pilot vehicles 
e Accurate perception/prediction is still work in progress 
e Transition from brute force data to safety case approach 
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EDGE CASE 

2021 Organizational Safety SLES Gre RESEARCH 

ct Significant pressure to deploy 
e Flurry of empty driver seat demos in late 2020 | 

e Can teams take the time needed for safety? | 
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= Industry transparency needed 
e Safety collaboration rather than competition 
e Public trust in face of an adverse news event 





https://youtu.be/nhqyrze30bk 
Yandex demo video 


= Ensuring robust safety cultures Ann Arbor Mi, Aug 2020 
e Silicon Valley culture + automotive culture + no human driver 
e We need to get this right to succeed! 
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